Explaining Variability in Electronic Health Record Effort in Primary Care Ambulatory Encounters.

BACKGROUND: Electronic health record (EHR) user interface event logs are fast providing another perspective on the value and efficiency EHR technology brings to health care. Analysis of these detailed usage data has demonstrated their potential to identify EHR and clinical process design factors related to user efficiency, satisfaction, and burnout. OBJECTIVE: This study aimed to analyze the event log data across 26 different health systems to determine the variability of use of a single vendor's EHR based on four event log metrics, at the individual, practice group, and health system levels. METHODS: We obtained de-identified event log data recorded from June 1, 2018, to May 31, 2019, from 26 health systems' primary care physicians. We estimated the variability in total Active EHR Time, Documentation Time, Chart Review Time, and Ordering Time across health systems, practice groups, and individual physicians. RESULTS: In total, 5,444 physicians (Family Medicine: 3,042 and Internal Medicine: 2,422) provided care in a total of 2,285 different practices nested in 26 health systems. Health systems explain 1.29, 3.55, 3.45, and 3.30% of the total variability in Active Time, Documentation Time, Chart Review Time, and Ordering Time, respectively. Practice-level variability was estimated to be 7.96, 13.52, 8.39, and 5.57%, respectively, and individual physicians explained the largest proportion of the variability for those same outcomes 17.09, 27.49, 17.51, and 19.75%, respectively. CONCLUSION: The most variable physician EHR usage patterns occurs at the individual physician level and decreases as you move up to the practice and health system levels. This suggests that interventions to improve individual users' EHR usage efficiency may have the most potential impact compared with those directed at health system or practice levels.

Physician Electronic Health Record Use After Changes in US Centers for Medicare & Medicaid Services Documentation Requirements.

This cohort study examines changes in physician electronic health record (EHR) documentation time before and after changes in Centers for Medicare & Medicaid evaluation and management requirements.

One-Year Adverse Outcomes Among US Adults With Post-COVID-19 Condition vs Those Without COVID-19 in a Large Commercial Insurance Database.

Importance: Many individuals experience ongoing symptoms following the onset of COVID-19, characterized as postacute sequelae of SARS-CoV-2 or post-COVID-19 condition (PCC). Less is known about the long-term outcomes for these individuals. Objective: To quantify 1-year outcomes among individuals meeting a PCC definition compared with a control group of individuals without COVID-19. Design, Setting, and Participants: This case-control study with a propensity score-matched control group included members of commercial health plans and used national insurance claims data enhanced with laboratory results and mortality data from the Social Security Administration's Death Master File and Datavant Flatiron data. The study sample consisted of adults meeting a claims-based definition for PCC with a 2:1 matched control cohort of individuals with no evidence of COVID-19 during the time period of April 1, 2020, to July 31, 2021. Exposures: Individuals experiencing postacute sequelae of SARS-CoV-2 using a Centers for Disease Control and Prevention-based definition. Main Outcomes and Measures: Adverse outcomes, including cardiovascular and respiratory outcomes and mortality, for individuals with PCC and controls assessed over a 12-month period. Results: The study population included 13 435 individuals with PCC and 26 870 individuals with no evidence of COVID-19 (mean [SD] age, 51 [15.1] years; 58.4% female). During follow-up, the PCC cohort experienced increased health care utilization for a wide range of adverse outcomes: cardiac arrhythmias (relative risk [RR], 2.35; 95% CI, 2.26-2.45), pulmonary embolism (RR, 3.64; 95% CI, 3.23-3.92), ischemic stroke (RR, 2.17; 95% CI, 1.98-2.52), coronary artery disease (RR, 1.78; 95% CI, 1.70-1.88), heart failure (RR, 1.97; 95% CI, 1.84-2.10), chronic obstructive pulmonary disease (RR, 1.94; 95% CI, 1.88-2.00), and asthma (RR, 1.95; 95% CI, 1.86-2.03). The PCC cohort also experienced increased mortality, as 2.8% of individuals with PCC vs 1.2% of controls died, implying an excess death rate of 16.4 per 1000 individuals. Conclusions and Relevance: This case-control study leveraged a large commercial insurance database and found increased rates of adverse outcomes over a 1-year period for a PCC cohort surviving the acute phase of illness. The results indicate a need for continued monitoring for at-risk individuals, particularly in the area of cardiovascular and pulmonary management.

Assessment of structured data elements for social risk factors.

OBJECTIVES: Computable social risk factor phenotypes derived from routinely collected structured electronic health record (EHR) or health information exchange (HIE) data may represent a feasible and robust approach to measuring social factors. This study convened an expert panel to identify and assess the quality of individual EHR and HIE structured data elements that could be used as components in future computable social risk factor phenotypes. STUDY DESIGN: Technical expert panel. METHODS: A 2-round Delphi technique included 17 experts with an in-depth knowledge of available EHR and/or HIE data. The first-round identification sessions followed a nominal group approach to generate candidate data elements that may relate to socioeconomics, cultural context, social relationships, and community context. In the second-round survey, panelists rated each data element according to overall data quality and likelihood of systematic differences in quality across populations (ie, bias). RESULTS: Panelists identified a total of 89 structured data elements. About half of the data elements (n = 45) were related to socioeconomic characteristics. The panelists identified a diverse set of data elements. Elements used in reimbursement-related processes were generally rated as higher quality. Panelists noted that several data elements may be subject to implicit bias or reflect biased systems of care, which may limit their utility in measuring social factors. CONCLUSIONS: Routinely collected structured data within EHR and HIE systems may reflect patient social risk factors. Identifying and assessing available data elements serves as a foundational step toward developing future computable social factor phenotypes.

Pediatrician Electronic Health Record Time Use for Outpatient Encounters.

BACKGROUND: The time providers spend using their electronic health records (EHRs) delivering care and its potential impact on patient care are of concern for the health care system. In studies to date, researchers have focused on providers who primarily care for adults. Scant information exists for pediatricians. Given this gap, it is important to quantify EHR activity for this group. METHODS: We studied pediatricians practicing in US-based ambulatory practices using the Cerner Millennium EHR by extracting data from software log files in the Lights On Network for the calendar year 2018 and summarizing the time spent on each of 13 clinically-focused EHR functions according to clinical specialty. RESULTS: Our data included >20 million encounters by almost 30 thousand physicians from 417 health systems. Pediatric physicians spent an average of 16 minutes per encounter using their EHR. Chart review (31%), documentation (31%), and ordering (13%) functions accounted for most of the time. The distribution of time spent by providers using their EHR is highly variable within subspecialty but is similar across specialties. Because of data limitations, we were unable to examine geographic or health system-specific variation. CONCLUSIONS: Pediatricians, like physicians who care for adults, spend a large portion of their day using their EHR. Additionally, although chart review and documentation accounted for 62% of the activity, as in previously published studies, in our study, we found that chart review accounted for half of that time. Wide variation suggests opportunities to optimize both the processes of entering information and searching for patient data within the EHR.

Prior presumed coronavirus infection reduces COVID-19 risk: A cohort study.

BACKGROUND: Immunological cross-reactivity between common cold coronaviruses (CCC) and SARS-CoV-2 might account for the reduced incidence of COVID-19 in children. Evidence to support speculation includes in vitro evidence for humoral and cellular cross-reactivity with SARS-CoV-2 in specimens obtained before the pandemic started. METHOD: We used retrospective health insurance enrollment records, claims, and laboratory results to assemble a cohort of 869,236 insured individuals who had a PCR test for SARS-CoV-2. We estimated the effects of having clinical encounters for various diagnostic categories in the year preceding the study period on the risk of a positive test result. FINDINGS: After adjusting for age, gender and care seeking behavior, we identified that individuals with diagnoses for common cold symptoms, including acute sinusitis, bronchitis, or pharyngitis in the preceding year had a lower risk of testing positive for SARS-CoV-2 (OR=0.76, 95%CI=0.75, 0.77). No reduction in the odds of a positive test for SARS-CoV-2 was seen in individuals under 18 years. The reduction in odds in adults remained stable for four years but was strongest in those with recent common cold symptoms. INTERPRETATION: While this study cannot attribute this association to cross-immunity resulting from a prior CCC infection, it is one potential explanation. Regardless of the cause, the reduction in the odds of being infected by SARS-CoV-2 among those with a recent diagnosis of common cold symptoms may have a role in shifting future COVD-19 infection patterns from endemic to episodic.

Physician Time Spent Using the Electronic Health Record During Outpatient Encounters: A Descriptive Study.

Background: The amount of time that providers spend using electronic health records (EHRs) to support the care delivery process is a concern for the U.S. health care system. Given the potential effect on patient care and the high costs related to this time, particularly for medical specialists whose work is largely cognitive, these findings warrant more precise documentation of the time physicians invest in these clinically focused EHR functions. Objective: To describe how much time ambulatory medical subspecialists and primary care physicians across several U.S. care delivery systems spend on various EHR functions. Design: Descriptive study. Setting: U.S.-based, adult, nonsurgical, ambulatory practices using the Cerner Millennium EHR. Participants: 155 000 U.S. physicians. Measurements: Data were extracted from software log files in the Lights On Network (Cerner) during 2018 that totaled the time spent on each of the 13 clinically focused EHR functions. Averages per encounter by specialty were computed. Results: This study included data from approximately 100 million patient encounters with about 155 000 physicians from 417 health systems. Physicians spent an average of 16 minutes and 14 seconds per encounter using EHRs, with chart review (33%), documentation (24%), and ordering (17%) functions accounting for most of the time. The distribution of time spent by providers using EHRs varies greatly within specialty. The proportion of time spent on various clinically focused functions was similar across specialties. Limitation: Variation by health system could not be examined, and all providers used the same software. Conclusion: The time spent using EHRs to support care delivery constitutes a large portion of the physicians' day, and wide variation suggests opportunities to optimize systems and processes. Primary Funding Source: None.

Technology-Enabled Clinical Trials: Transforming Medical Evidence Generation.

The complexity and costs associated with traditional randomized, controlled trials have increased exponentially over time, and now threaten to stifle the development of new drugs and devices. Nevertheless, the growing use of electronic health records, mobile applications, and wearable devices offers significant promise for transforming clinical trials, making them more pragmatic and efficient. However, many challenges must be overcome before these innovations can be implemented routinely in randomized, controlled trial operations. In October of 2018, a diverse stakeholder group convened in Washington, DC, to examine how electronic health record, mobile, and wearable technologies could be applied to clinical trials. The group specifically examined how these technologies might streamline the execution of clinical trial components, delineated innovative trial designs facilitated by technological developments, identified barriers to implementation, and determined the optimal frameworks needed for regulatory oversight. The group concluded that the application of novel technologies to clinical trials provided enormous potential, yet these changes needed to be iterative and facilitated by continuous learning and pilot studies.

Sepsis surveillance: an examination of parameter sensitivity and alert reliability.

OBJECTIVE: To examine performance of a sepsis surveillance system in a simulated environment where modifications to parameters and settings for identification of at-risk patients can be explored in-depth. MATERIALS AND METHODS: This was a multiple center observational cohort study. The study population comprised 14 917 adults hospitalized in 2016. An expert-driven rules algorithm was applied against 15.1 million data points to simulate a system with binary notification of sepsis events. Three system scenarios were examined: a scenario as derived from the second version of the Consensus Definitions for Sepsis and Septic Shock (SEP-2), the same scenario but without systolic blood pressure (SBP) decrease criteria (near SEP-2), and a conservative scenario with limited parameters. Patients identified by scenarios as being at-risk for sepsis were assessed for suspected infection. Multivariate binary logistic regression models estimated mortality risk among patients with suspected infection. RESULTS: First, the SEP-2-based scenario had a hyperactive, unreliable parameter SBP decrease >40 mm Hg from baseline. Second, the near SEP-2 scenario demonstrated adequate reliability and sensitivity. Third, the conservative scenario had modestly higher reliability, but sensitivity degraded quickly. Parameters differed in predicting mortality risk and represented a substitution effect between scenarios. DISCUSSION: Configuration of parameters and alert criteria have implications for patient identification and predicted outcomes. CONCLUSION: Performance of scenarios was associated with scenario design. A single hyperactive, unreliable parameter may negatively influence adoption of the system. A trade-off between modest improvements in alert reliability corresponded to a steep decline in condition sensitivity in scenarios explored.

A systematic comparison of contemporary automatic speech recognition engines for conversational clinical speech.

Conversations especially between a clinician and a patient are important sources of data to support clinical care. To date, clinicians act as the sensor to capture these data and record them in the medical record. Automatic speech recognition (ASR) engines have advanced to support continuous speech, to work independently of speaker and deliver continuously improving performance. Near human levels of performance have been reported for several ASR engines. We undertook a systematic comparison of selected ASRs for clinical conversational speech. Using audio recorded from unscripted clinical scenarios using two microphones, we evaluated eight ASR engines using word error rate (WER) and the precision, recall and F1 scores for concept extraction. We found a wide range of word errors across the ASR engines, with values ranging from 65% to 34%, all falling short of the rates achieved for other conversational speech. Recall for health concepts also ranged from 22% to 74%. Concept recall rates match or exceed expectations given measured word error rates suggesting that vocabulary is not the dominant issue.

The Impact of Censoring Drug Switching in Medication Adherence Measures of Chronic Single Ingredient Oral Drugs.

We explored how drug switching impacts adherence measures for common chronic oral medications. Switching between ingredients with the same indication was detected within a 30-day grace period. The proportion of days covered (PDC) and adherent status (cutoff 0.8) for each ingredient was calculated and compared between different censoring approaches: censoring drug switching (PDCswitch), censoring the end of dispensing (PDCend), and fixed 365-day period (PDC365). Overall, 854,380 (15.9%) patients in the Optum ClinFormatics (Optum) and 150,785 (22.0%) patients in the MarketScan Multi-state Medicaid (MDCD) had at least one switch within one year. Compared with PDC365 in Optum, PDCswitch means were higher: 0.85 vs. 0.41 for antihypertensive, 0.82 vs. 0.46 for antihyperglycemics, and 0.84 vs. 0.33 for antihyerlipidemia. Further, the percentages of adherent patients were higher: 95.8% vs. 17.9% for antihypertensive, 85.5% vs. 18.9% for antihyperglycemics, and 72.1% vs. 5.3% for antihyerlipidemia. Significant and modest changes were observed between PDCswitch and PDCend.

Factors Influencing Progress of Health Information Exchange Organizations in the United States.

Progress is being made toward improved healthcare interoperability in the United States, but exchange between electronic health records alone is insufficient. Using data from the eHealth Initiative's Annual Survey of Health Information Exchange, we developed models of HIE financial and operational progress. Our analysis suggests that organizations that focus on enabling exchange thorugh education and policy need to be considered separately from those focused on the actual exchange. The associations between characteristics and progress in data exchanging HIEs suggest that diversity of participants as both originators and receivers of data and breadth of data are important underlying success factors.

Nonadherence to Oral Antihyperglycemic Agents: Subsequent Hospitalization and Mortality among Patients with Type 2 Diabetes in Clinical Practice.

Using real-world clinical data from the Indiana Network for Patient Care, we analyzed the associations between non-adherence to oral antihyperglycemic agents (OHA) and subsequent diabetes-related hospitalization and all-cause mortality for patients with type 2 diabetes. OHA adherence was measured by the annual proportion of days covered (PDC) for 2008 and 2009. Among 24,067 eligible patients, 35,507 annual PDCs were formed. Over 90% (n=21,798) of the patients had a PDC less than 80%. In generalized linear mixed model analyses, OHA non-adherence is significantly associated with diabetes related hospitalizations (OR: 1.2; 95% CI [1.1,1.3]; p<0.0001). Older patients, white patients, or patients who had ischemic heart disease, stroke, or renal disease had higher odds of hospitalization. Similarly, OHA non-adherence increased subsequent mortality (OR: 1.3; 95% CI [1.02, 1.61]; p<0.0001). Patient age, male gender, income and presence of ischemic heart diseases, stroke, and renal disease were also significantly associated with subsequent all-cause death.

Bridging islands of information to establish an integrated knowledge base of drugs and health outcomes of interest.

The entire drug safety enterprise has a need to search, retrieve, evaluate, and synthesize scientific evidence more efficiently. This discovery and synthesis process would be greatly accelerated through access to a common framework that brings all relevant information sources together within a standardized structure. This presents an opportunity to establish an open-source community effort to develop a global knowledge base, one that brings together and standardizes all available information for all drugs and all health outcomes of interest (HOIs) from all electronic sources pertinent to drug safety. To make this vision a reality, we have established a workgroup within the Observational Health Data Sciences and Informatics (OHDSI, http://ohdsi.org) collaborative. The workgroup's mission is to develop an open-source standardized knowledge base for the effects of medical products and an efficient procedure for maintaining and expanding it. The knowledge base will make it simpler for practitioners to access, retrieve, and synthesize evidence so that they can reach a rigorous and accurate assessment of causal relationships between a given drug and HOI. Development of the knowledge base will proceed with the measureable goal of supporting an efficient and thorough evidence-based assessment of the effects of 1,000 active ingredients across 100 HOIs. This non-trivial task will result in a high-quality and generally applicable drug safety knowledge base. It will also yield a reference standard of drug-HOI pairs that will enable more advanced methodological research that empirically evaluates the performance of drug safety analysis methods.

A Comparison of Data Driven-based Measures of Adherence to Oral Hypoglycemic Agents in Medicaid Patients.

We evaluated and compared different methods for measuring adherence to Oral Antihyperglycemic Agents (OHA), based on the correlation between these measures and glycated hemoglobin A1C (HbA1c) levels in Medicaid patients with Type 2 diabetes. An observational sample of 831 Medicaid patients with Type 2 diabetes who had HbA1c test results recorded between January 1, 2001 and December 31, 2005 was identified in the Indiana Network of Patient Care (INPC). OHA adherence was measured by medication possession ratio (MPR), proportion of days covered (PDC), and the number of gaps (GAP) for 3, 6, and 12-month intervals prior to the HbA1c test date. All three OHA adherence measurements showed consistent and significant correlation with HbA1c level. The 6-month PDC showed the strongest association with HbA1c levels in both unadjusted (-1.07, P<0.0001) and adjusted (-1.12, P<0.0001) models.

Desideratum for evidence based epidemiology.

BACKGROUND: There is great variation in choices of method and specific analytical details in epidemiological studies, resulting in widely varying results even when studying the same drug and outcome in the same database. Not only does this variation undermine the credibility of the research but it limits our ability to improve the methods. METHODS: In order to evaluate the performance of methods and analysis choices we used standard references and a literature review to identify 164 positive controls (drug-outcome pairs believed to represent true adverse drug reactions), and 234 negative controls (drug-outcome pairs for which we have confidence there is no direct causal relationship). We tested 3,748 unique analyses (methods in combination with specific analysis choices) that represent the full range of approaches to adjusting for confounding in five large observational datasets on these controls. We also evaluated the impact of increasingly specific outcome definitions, and performed a replication study in six additional datasets. We characterized the performance of each method using the area under the receiver operator curve (AUC), bias, and coverage probability. In addition, we developed simulated datasets that closely matched the characteristics of the observational datasets into which we inserted data consistent with known drug-outcome relationships in order to measure the accuracy of estimates generated by the analyses. DISCUSSION: We expect the results of this systematic, empirical evaluation of the performance of these analyses across a moderate range of outcomes and databases to provide important insights into the methods used in epidemiological studies and to increase the consistency with which methods are applied, thereby increasing the confidence in results and our ability to systematically improve our approaches.

Variation in choice of study design: findings from the Epidemiology Design Decision Inventory and Evaluation (EDDIE) survey.

BACKGROUND: Researchers using observational data to understand drug effects must make a number of analytic design choices that suit the characteristics of the data and the subject of the study. Review of the published literature suggests that there is a lack of consistency even when addressing the same research question in the same database. OBJECTIVE: To characterize the degree of similarity or difference in the method and analysis choices made by observational database research experts when presented with research study scenarios. RESEARCH DESIGN: On-line survey using research scenarios on drug-effect studies to capture method selection and analysis choices that follow a dependency branching based on response to key questions. SUBJECTS: Voluntary participants experienced in epidemiological study design solicited for participation through registration on the Observational Medical Outcomes Partnership website, membership in particular professional organizations, or links in relevant newsletters. MEASURES: Description (proportion) of respondents selecting particular methods and making specific analysis choices based on individual drug-outcome scenario pairs. The number of questions/decisions differed based on stem questions of study design, time-at-risk, outcome definition, and comparator. RESULTS: There is little consistency across scenarios, by drug or by outcome of interest, in the decisions made for design and analyses in scenarios using large healthcare databases. The most consistent choice was the cohort study design but variability in the other critical decisions was common. CONCLUSIONS: There is great variation among epidemiologists in the design and analytical choices that they make when implementing analyses in observational healthcare databases. These findings confirm that it will be important to generate empiric evidence to inform these decisions and to promote a better understanding of the impact of standardization on research implementation.

Managing data quality for a drug safety surveillance system.

OBJECTIVE: The objective of this study is to present a data quality assurance program for disparate data sources loaded into a Common Data Model, highlight data quality issues identified and resolutions implemented. BACKGROUND: The Observational Medical Outcomes Partnership is conducting methodological research to develop a system to monitor drug safety. Standard processes and tools are needed to ensure continuous data quality across a network of disparate databases, and to ensure that procedures used to extract-transform-load (ETL) processes maintain data integrity. Currently, there is no consensus or standard approach to evaluate the quality of the source data, or ETL procedures. METHODS: We propose a framework for a comprehensive process to ensure data quality throughout the steps used to process and analyze the data. The approach used to manage data anomalies includes: (1) characterization of data sources; (2) detection of data anomalies; (3) determining the cause of data anomalies; and (4) remediation. FINDINGS: Data anomalies included incomplete raw dataset: no race or year of birth recorded. Implausible data: year of birth exceeding current year, observation period end date precedes start date, suspicious data frequencies and proportions outside normal range. Examples of errors found in the ETL process were zip codes incorrectly loaded, drug quantities rounded, drug exposure length incorrectly calculated, and condition length incorrectly programmed. CONCLUSIONS: Complete and reliable observational data are difficult to obtain, data quality assurance processes need to be continuous as data is regularly updated; consequently, processes to assess data quality should be ongoing and transparent.